Characterizing and improving the fault tolerance of artificial neural networks

Artificial neural networks are networks of very simple processing elements based on an approximate model of a biological neuron. It is widely believed that because biological neural networks are tolerant of the loss of individual neurons and because there is a strong analogy between biological neural networks and artificial neural networks, then artificial neural networks must also be inherently fault tolerant. This is, unfortunately, simply not true. Results reported in this dissertation show that in the task of function approximation the multilayer perceptron is very intolerant of faults to the extent that the loss of a single network parameter can ruin the learned approximation. A method for quantitatively evaluating network fault tolerance is proposed in this dissertation. This method is applicable to a large number of artificial neural network architectures. Using this method, it can be shown that the generalized radial basis function network is much more fault tolerant than the multilayer perceptron. Additionally, the generalized radial basis function network learns more rapidly than the multilayer perceptron. These findings can be explained by using spectral methods. When the spectral content of a network\u27s activation function is not similar to the spectral content of the function to be learned, then learning is made very difficult and the learned solution is very sensitive to the loss or alteration of network parameters. Numerous methods for improving the fault tolerance of artificial neural networks are presented and discussed. Interestingly, experimental results show that a well chosen set of initial conditions can measurably improve fault tolerance. Also, training with random intermittent faults can significantly improve the fault tolerance of neural networks. In fact, the improvement in fault tolerance in the generalized radial basis function network was such that the loss of any single weight actually caused improvement or no change from the fault-free performance. Finally, this dissertation presents guidelines for intelligently choosing network parameters for good learning as well as improved fault tolerance

REU Site: Supercomputing Undergraduate Program in Maine (SuperMe)

This award, for a new Research Experience for Undergraduates (REU) site, builds a Supercomputing Undergraduate Program in Maine (SuperMe). This new site provides ten-week summer research experiences at the University of Maine (UMaine) for ten undergraduates each year for three years. With integrated expertise of ten faculty researchers from both computer systems and domain applications, SuperMe allows each undergraduate to conduct meaningful research, such as developing supercomputing techniques and tools, and solving cutting-edge research problems through parallel computing and scientific visualization. Besides being actively involved in research groups, students attend weekly seminars given by faculty mentors, formally report and present their research experiences and results, conduct field trips, and interact with ITEST, RET and GK-12 participants. SuperMe provides scientific exploration ranging from engineering to sciences with a coherent intellectual focus on supercomputing. It consists of four computer systems projects that aim to improve techniques in grid computing, parallel I/O data accesses, high-resolution scientific visualization and information security, and five computer modeling projects that utilize world-class supercomputing and visualization facilities housed at UMaine to perform large, complex simulation experiments and data analysis in different science domains. SuperMe provides a diversity of cutting-edge research opportunities to students from under-represented groups or from universities in rural areas with limited research opportunities. Through interacting directly with the participant of existing programs at UMaine, including ITEST, RET and GK-12, REU students disseminates their research results and experiences to middle and high school students and teachers. This site is co-funded by the Department of Defense in partnership with the NSF REU Site program

IDEAS: Inquiry-based Dynamic Earth Applications of Supercomputing, Seeing the Big Picture with Information Technology

This project will connect researchers at the University of Maine with students and middle school teachers, both at the University (during a summer workshop) and at participating schools (during the academic year) to utilize computer modeling and visualization of geological processes in the classroom. The proposed project will have 60 participating teachers each with 120 contact hours at the University of Maine, as well as 180 students, each with 20 contact hours at the University of Maine. The focus of this project is to integrate computational modeling with the existing science curriculum at the middle school level. This will be accomplished largely by collectively utilizing existing laptop computer computational power and networking capability to run computer models, both locally and at the University supercomputer, and to create high resolution interactive visualization displays (from the same laptops) to view the output. The specific goals are to: 1) develop numeric simulation and visualization tools for geodynamics with the involvement of middle school teachers and education experts; 2) train middle school teachers on the integration of such tools in the teaching of the existing curriculum topics; 3) stimulate middle school students\u27 interest in science and technology and improve their knowledge and performance in these areas; and 4) disseminate such tools and effective pedagogies enabled by them to all middle schools in Maine, with the promise of the tools and methods serving as a model for other schools contemplating the use of laptop computers in the classroom. The project takes advantage of the fact that every seventh and eighth grade student and teacher in the state of Maine is issued an Apple laptop computer. These computers are all networked together and to the outside world via wireless networks within the classroom. Additionally, all schools and libraries within the state of Maine have high speed (wired) Internet connections. Another factor that is leveraged is the University of Maine\u27s 512 CPU cluster supercomputer that is also based on the Apple platform, as well as a number of researchers who perform numerical modeling using this cluster

IDEAS: Inquiry-based Dynamic Earth Applications of Supercomputing, Seeing the Big Picture with Information Technology

The goals of the project were to increase the level and volume of information technologies in the classroom and to promote inquiry-based learning. The project was tightly integrated with the Maine Learning Technology Initiative that puts a laptop computer into the hands of every 7th and 8th grade student and teacher. It was also tightly integrated with the University of Maine Supercomputer. Through the use of technology, students were able to ask “what if questions and find and visualize the answers to their questions. The focus of the inquiry was dynamic Earth modelling. This included geological evolution of the earth as well as weather and climate changes overtime

Maine EPSCoR End-to-End Connectivity for Sustainability Science Collaboration

This NSF EPSCoR C2 project allowed Maine EPSCoR to continue the state’s momentum to enhance the connectivity of the state’s research, higher education, and K-12 institutions through Maine’s Research and Education Network (MaineREN). Over the last few years, multi-million dollar investments have built networking and computing power at the state level, including: 1) the installation of 1,100 miles of middle-mile fiber optic cable; 2) investments in shared computing resources for high performance computing and cloud computing; 3) the Maine School and Library Network; 4) the Maine Learning Technology Initiative (grade 6-12 laptops); and 5) investments in high-performance visualization and videoconferencing. This C2 project allowed Maine EPSCoR to address the cyberinfrastructure gaps at the seven campuses of the University of Maine System that had still been preventing the delivery of true end-to-end connectivity between Maine’s researchers and the new advanced networking services provided over MaineREN. The research and education focus that was enabled by this C2 project is the Maine EPSCoR Sustainability Science Initiative (SSI) Rll Track 1, with the goal of providing SSI researchers and students at the seven campuses of the University of Maine System true end-to-end connectivity. Cyberinfrastructure is an important key to helping SSI to advance their sustainability science objectives to: 1) examine interactions between social and ecological systems (SES) as landscapes change in response to urbanization, forest management, and climate variability; 2) investigate how such SES knowledge affects, and is influenced by, the actions and decisions of diverse stakeholders, with a goal of strengthening connections between knowledge and action; 3) evaluate the factors that facilitate and impede interdisciplinary collaboration, with a goal of identifying and implementing individual and institutional best practices that are needed to support successful interdisciplinary research programs in sustainability science. In particular, the C2 connectivity improvements that are now in place will support the Track 1 SSI research agenda by addressing various data management, visualization, and virtual proximity challenges that were present. Except for a small amount of support towards the AAAS review, all of the C2 budget was allocated for the capital cyberinfrastructure improvements, with the goal of enabling the effectiveness of the research and education activities of the SSI Track 1 project. This then means that there is a high degree of leveraging and synergy between the two projects, and that the personnel participation, research, diversity, and workforce development activities were supported from a variety of other sources including SSI Track 1, state funds, university funds, and UMaine System funds (and therefore are not a direct part of this award). While somewhat confusing for reporting purposes, this high degree of leveraging resulted in a tightly integrated and effective manner of furthering Maine’s research and education capacity in Sustainability Science. The implementation and administration of all three NSF EPSCoR projects (Track 1, 2, C2) has been through the Maine EPSCoR office at the University of Maine, which allowed for effective coordination and leveraging of resources and investments for the maximum benefit to Maine researchers

Maine EPSCoR End-to-End Connectivity for Sustainability Science Collaboration

Project DescriptionThis RII C2 proposal from Maine (ME) EPSCoR is focused on addressing last-mile bottlenecks at seven campuses of the University of Maine System. Maine\u27s Research and Education Network, MaineREN, delivers high performance inter-campus fiber connectivity to public and private institutions across the state, but the intra-campus networking has lacked the same investment by the state.The proposed improvements include:- Rewiring eight buildings at the University of Maine Orono Campus (UMaine) with Cat-6 cable, increasing end-to-end performance to 10 Gbps.- Upgrading the fiber backbone between the two University of Southern Maine (USM) campuses, one in Portland and one in Gorham, 12 miles apart. In addition, upgrades will be done for the buildings housing the ME RII Track-1 researchers, including the Law Building, Library, Bailey Hall, and the buildings that make up the fiber core for the Portland campus. - Upgrades to edge routers to connect to the MaineREN backbone for UMaine Augusta (UMA), UMaine Farmington (UMF), UMaine Fort Kent (UMFK), UMaine Machias (UMM), and UMaine Presque Isle (UMPI). Intellectual MeritThe proposed upgrades in network connections will greatly improve the networking capacity available to the University of Maine system and enable researchers to take advantage of state-wide upgrades with improved end-to-end performance. The proposed RII C2 connectivity improvements will support the Maine RII Track-1 Sustainability Science Initiative (SSI) by increasing bandwidth availability for the SSI data management and visualization approaches. SSI is advancing the emerging field of sustainability science in three integrative ways: 1) examining interactions between social and ecological systems (SES) as landscapes change in response to urbanization, forest management, and climate variability; 2) investigating how much SES knowledge affects, and is influenced by, the actions and decision of stakeholders, with a goal of strengthening connections between knowledge and actions; 3) evaluating the factors that facilitate and impede interdisciplinary collaboration, with a goal of identifying and implementing individual and institutional best practices that are needed to support successful interdisciplinary research programs in sustainability science.Broader ImpactsBy filling in relatively small gaps in the infrastructure, Maine will be able to make very large gains in the effectiveness of the state\u27s cyberinfrastructure (CI) that will allow researchers to fully utilize investments to improve research effectiveness, promote collaboration, improve K-12 interaction, and develop the future workforce of the state. The networking upgrades will support the 300 researchers, students, and stakeholders that are part of the SSI collaboration over 17 different disciplinary fields. The SSI activities have the potential to increase Maine\u27s research capacity and competitiveness and grow Maine\u27s green innovation economy. The proposed project will leverage the RII Track-1 programs for broader impacts

MRI: Acquisition of Interactive Visualization Tools for Supercomputer Models

This project, acquiring a visualization facility (vizwall with high resolution display and high volume storage system to visualize large size data generated from diverse research activities), models polar ice sheets, oceans, atmospheric turbulent boundary layers, and geodynamics. The facility, whose main components consist of a visualization wall, a PRISM visualization server, and RAID storage disks, will be integrated to the university\u27s existing supercomputer cluster

MRI: Acquisition of a High Performance Cluster for the University of Maine Scientific Grid Portal

This project, acquiring a cluster to establish a scientific grid portal in Maine, aims to enable projects requiring large datasets. The work makes available to the wider community results such as widely-used whole-ice sheet models, tools for climate change research, prototype versions of object-based caching system (bundled with MPI-IO implementation developed at Argonne National Lab), the data management system, real-time animations, videos, etc. Additionally, the portal provides the larger community the compute power, storage capacity, and rendering engine to execute very high-resolution models, and receive animations and other visualized information in real time.Broader Impact: The infrastructure enhances understanding of global issues and contributes in the development of educational tools for K-12 students. The scientific grid portal contributes in the dissemination of important scientific discoveries. The portal also provides a show-case for research being performed in the state

Undergraduate Research Participation in Electrical Engineering

During the 1990-2003 summers the Electrical Engineering Department at the University of Maine will offer ten undergraduate students the opportunity to actively participate in research. Students will receive financial awards plus a subsistence allowance. The available research projects include (1) Environmental Sensors; (2) Intelligent Systems for Automation; (3) Communications Devices and Applications; (4) Motion Control; (5) Microprocessor/Instrumentation Applications; (6) Growth and Characterization of Thin Film Materials; and (7) Power Systems Applications. At least five students will come from institutions where research opportunities are limited and at least four students will be women, minorities or students with disabilities. Students chosen for the program will have displayed a high degree of initiative and independence of thought in both laboratories and course work. Student research projects are chosen to match the student\u27s interest and educational level. In addition to extensive University facilities, students will also have access to facilities at various nearby industries such as Sensor Research and Development Corporation, BIODE Corporation, Bangor Hydro Electric and Central Maine Power Companies, James River, Champion, and Scott Paper Companies, Digital Equipment Corporation, Fairchild and National Semiconductor. At the program culmination a written report and an oral seminar are required from the student. Three academic credits are awarded to the student upon satisfactory completion of the program